Pandas Profiling Report

Dataset statistics

Number of variables	8
Number of observations	1552
Missing cells	0
Missing cells (%)	0.0%
Duplicate rows	0
Duplicate rows (%)	0.0%
Total size in memory	91.1 KiB
Average record size in memory	60.1 B

Variable types

Numeric	8

Alerts

`profit` is highly correlated with `recencydays` and 1 other fields	High correlation
`recencydays` is highly correlated with `profit`	High correlation
`qtd_items` is highly correlated with `profit` and 1 other fields	High correlation
`avg_basket_size` is highly correlated with `qtd_items`	High correlation
`profit` is highly correlated with `qtd_items`	High correlation
`qtd_items` is highly correlated with `profit` and 1 other fields	High correlation
`avg_ticket` is highly correlated with `qtd_items`	High correlation
`profit` is highly correlated with `qtd_items`	High correlation
`qtd_items` is highly correlated with `profit`	High correlation
`df_index` is highly correlated with `recencydays`	High correlation
`profit` is highly correlated with `qtd_items` and 1 other fields	High correlation
`recencydays` is highly correlated with `df_index`	High correlation
`qtd_items` is highly correlated with `profit` and 1 other fields	High correlation
`avg_ticket` is highly correlated with `profit` and 1 other fields	High correlation
`avg_ticket` is highly skewed (γ1 = 27.98323513)	Skewed
`df_index` has unique values	Unique
`customerid` has unique values	Unique
`recencydays` has 25 (1.6%) zeros	Zeros

Reproduction

Analysis started	2022-09-23 09:03:35.618123
Analysis finished	2022-09-23 09:04:10.743121
Duration	35.12 seconds
Software version	pandas-profiling v3.1.0
Download configuration	config.json

df_index
Real number (ℝ_≥0)

HIGH CORRELATION
UNIQUE

Distinct	1552
Distinct (%)	100.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	2664.299613

Minimum	0
Maximum	7757
Zeros	1
Zeros (%)	0.1%
Negative	0
Negative (%)	0.0%
Memory size	12.2 KiB

Quantile statistics

Minimum	0
5-th percentile	176.55
Q1	920.75
median	2175.5
Q3	4135.75
95-th percentile	6536.05
Maximum	7757
Range	7757
Interquartile range (IQR)	3215

Descriptive statistics

Standard deviation	2057.528605
Coefficient of variation (CV)	0.7722587184
Kurtosis	-0.6910188006
Mean	2664.299613
Median Absolute Deviation (MAD)	1440.5
Skewness	0.6626497421
Sum	4134993
Variance	4233423.96
Monotonicity	Strictly increasing

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	1	0.1%
3190	1	0.1%
3385	1	0.1%
3384	1	0.1%
3378	1	0.1%
3368	1	0.1%
3362	1	0.1%
3359	1	0.1%
3355	1	0.1%
3328	1	0.1%
Other values (1542)	1542	99.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	1	0.1%
1	1	0.1%
2	1	0.1%
4	1	0.1%
5	1	0.1%
6	1	0.1%
7	1	0.1%
8	1	0.1%
12	1	0.1%
14	1	0.1%

Value	Count	Frequency (%)
7757	1	0.1%
7678	1	0.1%
7620	1	0.1%
7597	1	0.1%
7553	1	0.1%
7510	1	0.1%
7504	1	0.1%
7501	1	0.1%
7499	1	0.1%
7472	1	0.1%

customerid
Real number (ℝ_≥0)

UNIQUE

Distinct	1552
Distinct (%)	100.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	15179.49742

Minimum	12346
Maximum	18282
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	6.2 KiB

Quantile statistics

Minimum	12346
5-th percentile	12584.55
Q1	13701.5
median	15123.5
Q3	16662.25
95-th percentile	17976.8
Maximum	18282
Range	5936
Interquartile range (IQR)	2960.75

Descriptive statistics

Standard deviation	1723.792714
Coefficient of variation (CV)	0.113560592
Kurtosis	-1.174294211
Mean	15179.49742
Median Absolute Deviation (MAD)	1493
Skewness	0.09372121084
Sum	23558580
Variance	2971461.322
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
17850	1	0.1%
12949	1	0.1%
13995	1	0.1%
14619	1	0.1%
13005	1	0.1%
15754	1	0.1%
13950	1	0.1%
14912	1	0.1%
18272	1	0.1%
13771	1	0.1%
Other values (1542)	1542	99.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
12346	1	0.1%
12352	1	0.1%
12359	1	0.1%
12362	1	0.1%
12365	1	0.1%
12375	1	0.1%
12379	1	0.1%
12380	1	0.1%
12381	1	0.1%
12383	1	0.1%

Value	Count	Frequency (%)
18282	1	0.1%
18277	1	0.1%
18276	1	0.1%
18274	1	0.1%
18272	1	0.1%
18270	1	0.1%
18269	1	0.1%
18268	1	0.1%
18263	1	0.1%
18260	1	0.1%

profit
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION

Distinct	1550
Distinct (%)	99.9%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	4647.002965

Minimum	12.4
Maximum	336942.1
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	12.2 KiB

Quantile statistics

Minimum	12.4
5-th percentile	279.066
Q1	750.48
median	1656.615
Q3	3550.115
95-th percentile	11690.5345
Maximum	336942.1
Range	336929.7
Interquartile range (IQR)	2799.635

Descriptive statistics

Standard deviation	17274.9834
Coefficient of variation (CV)	3.717446174
Kurtosis	184.1829746
Mean	4647.002965
Median Absolute Deviation (MAD)	1078.165
Skewness	12.24371597
Sum	7212148.602
Variance	298425051.3
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
398.3	2	0.1%
2430.04	2	0.1%
2887.29	1	0.1%
461.47	1	0.1%
761.56	1	0.1%
894.18	1	0.1%
3092.38	1	0.1%
2637.2	1	0.1%
1525.31	1	0.1%
1545.24	1	0.1%
Other values (1540)	1540	99.2%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
12.4	1	0.1%
21.95	1	0.1%
26.6	1	0.1%
39.8	1	0.1%
51	1	0.1%
63.45	1	0.1%
64.25	1	0.1%
70.23	1	0.1%
78.15	1	0.1%
93.43	1	0.1%

Value	Count	Frequency (%)
336942.1	1	0.1%
280923.02	1	0.1%
262876.11	1	0.1%
201619.41	1	0.1%
155077.5	1	0.1%
154367.2	1	0.1%
126103.61	1	0.1%
121375.12	1	0.1%
111057.07	1	0.1%
93999.38	1	0.1%

recencydays
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
ZEROS

Distinct	244
Distinct (%)	15.7%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	64.96778351

Minimum	0
Maximum	373
Zeros	25
Zeros (%)	1.6%
Negative	0
Negative (%)	0.0%
Memory size	12.2 KiB

Quantile statistics

Minimum	0
5-th percentile	1
Q1	10
median	28
Q3	78
95-th percentile	274
Maximum	373
Range	373
Interquartile range (IQR)	68

Descriptive statistics

Standard deviation	84.65967212
Coefficient of variation (CV)	1.303102362
Kurtosis	2.673103827
Mean	64.96778351
Median Absolute Deviation (MAD)	24
Skewness	1.843840174
Sum	100830
Variance	7167.260083
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
1	63	4.1%
2	59	3.8%
8	50	3.2%
3	49	3.2%
4	47	3.0%
22	36	2.3%
16	35	2.3%
17	35	2.3%
10	34	2.2%
7	34	2.2%
Other values (234)	1110	71.5%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	25	1.6%
1	63	4.1%
2	59	3.8%
3	49	3.2%
4	47	3.0%
5	21	1.4%
7	34	2.2%
8	50	3.2%
9	30	1.9%
10	34	2.2%

Value	Count	Frequency (%)
373	2	0.1%
372	3	0.2%
371	1	0.1%
369	1	0.1%
368	1	0.1%
366	3	0.2%
365	3	0.2%
364	1	0.1%
360	1	0.1%
359	1	0.1%

qtd_items
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION

Distinct	619
Distinct (%)	39.9%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	565.7048969

Minimum	1
Maximum	80996
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	12.2 KiB

Quantile statistics

Minimum	1
5-th percentile	38
Q1	100
median	183
Q3	346.25
95-th percentile	1298.7
Maximum	80996
Range	80995
Interquartile range (IQR)	246.25

Descriptive statistics

Standard deviation	3206.037822
Coefficient of variation (CV)	5.667332631
Kurtosis	448.8159632
Mean	565.7048969
Median Absolute Deviation (MAD)	103.5
Skewness	19.64767536
Sum	877974
Variance	10278678.51
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
52	15	1.0%
70	13	0.8%
106	12	0.8%
60	12	0.8%
118	10	0.6%
87	10	0.6%
117	10	0.6%
189	9	0.6%
120	9	0.6%
66	9	0.6%
Other values (609)	1443	93.0%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
1	1	0.1%
2	1	0.1%
3	1	0.1%
4	1	0.1%
6	1	0.1%
9	2	0.1%
11	1	0.1%
12	5	0.3%
15	1	0.1%
16	3	0.2%

Value	Count	Frequency (%)
80996	1	0.1%
74215	1	0.1%
38639	1	0.1%
17376	1	0.1%
17150	1	0.1%
16288	1	0.1%
15853	1	0.1%
13369	1	0.1%
12872	1	0.1%
10828	1	0.1%

avg_ticket
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
SKEWED

Distinct	1550
Distinct (%)	99.9%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	130.6282729

Minimum	2.241
Maximum	77183.6
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	12.2 KiB

Quantile statistics

Minimum	2.241
5-th percentile	6.159015874
Q1	15.48176257
median	19.12091844
Q3	26.80088617
95-th percentile	95.14292179
Maximum	77183.6
Range	77181.359
Interquartile range (IQR)	11.31912361

Descriptive statistics

Standard deviation	2447.777236
Coefficient of variation (CV)	18.73849498
Kurtosis	810.7275853
Mean	130.6282729
Median Absolute Deviation (MAD)	4.969503921
Skewness	27.98323513
Sum	202735.0796
Variance	5991613.398
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
16.83333333	2	0.1%
25.5	2	0.1%
15.289	1	0.1%
19.75769231	1	0.1%
16.3775	1	0.1%
21.28742857	1	0.1%
32.14259259	1	0.1%
18.54566265	1	0.1%
22.13823529	1	0.1%
73.7905	1	0.1%
Other values (1540)	1540	99.2%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
2.241	1	0.1%
2.264375	1	0.1%
2.817681159	1	0.1%
3.1	1	0.1%
3.140802469	1	0.1%
3.157113402	1	0.1%
3.269333333	1	0.1%
3.45	1	0.1%
3.487294118	1	0.1%
3.734567219	1	0.1%

Value	Count	Frequency (%)
77183.6	1	0.1%
56157.5	1	0.1%
13305.5	1	0.1%
4453.43	1	0.1%
2027.86	1	0.1%
952.9875	1	0.1%
931.5	1	0.1%
835.864	1	0.1%
643.8585714	1	0.1%
602.4531323	1	0.1%

frequency
Real number (ℝ_≥0)

Distinct	873
Distinct (%)	56.2%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	0.2001652599

Minimum	0.005479452055
Maximum	17
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	12.2 KiB

Quantile statistics

Minimum	0.005479452055
5-th percentile	0.01078465562
Q1	0.01953125
median	0.03109215265
Q3	0.06598173516
95-th percentile	1
Maximum	17
Range	16.99452055
Interquartile range (IQR)	0.04645048516

Descriptive statistics

Standard deviation	0.5742324675
Coefficient of variation (CV)	2.868791856
Kurtosis	474.116048
Mean	0.2001652599
Median Absolute Deviation (MAD)	0.01509215265
Skewness	17.00132451
Sum	310.6564834
Variance	0.3297429267
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
1	216	13.9%
0.02941176471	9	0.6%
0.03703703704	8	0.5%
0.08333333333	8	0.5%
0.01923076923	7	0.5%
0.02	7	0.5%
0.02173913043	7	0.5%
2	7	0.5%
0.02857142857	6	0.4%
0.02409638554	6	0.4%
Other values (863)	1271	81.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0.005479452055	1	0.1%
0.005681818182	1	0.1%
0.005714285714	1	0.1%
0.00583090379	1	0.1%
0.005899705015	1	0.1%
0.005952380952	2	0.1%
0.006006006006	1	0.1%
0.006451612903	1	0.1%
0.00651465798	1	0.1%
0.006600660066	1	0.1%

Value	Count	Frequency (%)
17	1	0.1%
4	1	0.1%
3	2	0.1%
2	7	0.5%
1.142857143	1	0.1%
1	216	13.9%
0.75	1	0.1%
0.6666666667	1	0.1%
0.550802139	1	0.1%
0.5335120643	1	0.1%

avg_basket_size
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	1283
Distinct (%)	82.7%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	0.008494577938

Minimum	1.347436502 × 10^-5
Maximum	0.6666666667
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	12.2 KiB

Quantile statistics

Minimum	1.347436502 × 10^-5
5-th percentile	0.001460699064
Q1	0.003302374003
median	0.005154639175
Q3	0.008443806649
95-th percentile	0.01967629123
Maximum	0.6666666667
Range	0.6666531923
Interquartile range (IQR)	0.005141432646

Descriptive statistics

Standard deviation	0.0267436409
Coefficient of variation (CV)	3.148318974
Kurtosis	388.5742713
Mean	0.008494577938
Median Absolute Deviation (MAD)	0.00225742778
Skewness	18.49588505
Sum	13.18358496
Variance	0.0007152223284
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0.008771929825	6	0.4%
0.005263157895	5	0.3%
0.01162790698	5	0.3%
0.003472222222	5	0.3%
0.01219512195	5	0.3%
0.004807692308	4	0.3%
0.004854368932	4	0.3%
0.005291005291	4	0.3%
0.007246376812	4	0.3%
0.007352941176	4	0.3%
Other values (1273)	1506	97.0%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
1.347436502 × 10^-5	1	0.1%
2.469227255 × 10^-5	1	0.1%
0.0001664078101	1	0.1%
0.000270374662	1	0.1%
0.0003747006193	1	0.1%
0.0004580432393	1	0.1%
0.0004628915291	1	0.1%
0.0004669624095	1	0.1%
0.0004802553099	1	0.1%
0.0005119017149	1	0.1%

Value	Count	Frequency (%)
0.6666666667	1	0.1%
0.5	2	0.1%
0.25	1	0.1%
0.1875	1	0.1%
0.1627906977	1	0.1%
0.08333333333	2	0.1%
0.07692307692	1	0.1%
0.05555555556	1	0.1%
0.05263157895	1	0.1%
0.04529616725	1	0.1%

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

Count
Matrix

A simple visualization of nullity by column.

Nullity matrix is a data-dense display which lets you quickly visually pick out patterns in data completion.

First rows

	df_index	customerid	profit	recencydays	qtd_items	avg_ticket	frequency	avg_basket_size
0	0	17850	5493.79	372.0	35.0	18.152222	17.000000	0.019619
1	1	13047	3395.98	31.0	132.0	18.822907	0.028302	0.007189
2	2	12583	7375.42	2.0	1569.0	29.479271	0.040323	0.002964
3	4	15100	1116.90	333.0	48.0	292.000000	0.073171	0.037500
4	5	15291	4740.09	25.0	508.0	45.323301	0.040115	0.007133
5	6	14688	6154.36	7.0	579.0	17.219786	0.057221	0.005800
6	7	17809	6196.20	16.0	961.0	88.719836	0.033520	0.005834
7	8	15311	62116.46	0.0	2167.0	25.543464	0.243316	0.002383
8	12	16029	111057.07	38.0	10828.0	334.813388	0.184524	0.001567
9	14	12431	6558.51	35.0	1130.0	27.489195	0.044248	0.005141

Last rows

	df_index	customerid	profit	recencydays	qtd_items	avg_ticket	frequency	avg_basket_size
1542	7472	15877	545.78	1.0	177.0	3.823876	0.117647	0.005391
1543	7499	12586	213.94	17.0	56.0	17.903636	1.000000	0.012658
1544	7501	16376	996.50	8.0	276.0	7.896080	0.200000	0.002882
1545	7504	12452	432.57	16.0	95.0	19.571364	1.000000	0.010811
1546	7510	18084	93.43	16.0	312.0	90.480000	1.000000	0.003205
1547	7553	17727	1077.95	15.0	111.0	16.064394	1.000000	0.001550
1548	7597	12479	577.10	11.0	87.0	17.006452	1.000000	0.002597
1549	7620	14126	768.63	7.0	361.0	47.075333	0.750000	0.005906
1550	7678	12558	539.92	7.0	102.0	24.541818	1.000000	0.005102
1551	7757	14087	207.17	2.0	113.0	2.817681	1.000000	0.003984

Overview

Variables

Interactions

Correlations

Spearman's ρ

Pearson's r

Kendall's τ

Phik (φk)

Missing values

Sample

First rows

Last rows